User-aware video display

ABSTRACT

The present invention provides methods and systems for displaying a video signal depending upon a user interaction. An example system includes a display for receiving the video signal, at least one transmitter for transmitting location signals, and at least three transceivers for receiving the location signals from the transmitter and for transmitting modified location signals. The geometric locations of the at least three transceivers are known to a computing device. The computing device is then able to derive a transmit location of the location signals based on the received modified location signals, wherein the computing device is adapted to modify the content of the video signal in response to the derived transmit location depending upon the user interaction.

FIELD OF INVENTION

This invention relates to a method and system for displaying a videosignal on a display depending upon a user interaction. The providedsystem is particularly well suited for, but not limited to, wirelesscommunication and positioning technologies exploiting the ultra-wideband(UWB) radio channel recently made available by the US FederalCommunications Commission (FCC) in the frequency range between 3.1 GHzand 10.6 GHz.

BACKGROUND

Short-range wireless technologies in the wireless local area network(WLAN) space as well as wireless personal and body area networks (WPANand WBAN) continue to proliferate rapidly. Similarly, wirelesstechnology is increasingly applied to feed high-resolution video signalsover a digital video interface (DVI) to hand-held or detachable flatpanel displays (FPDs) as well as to stationary displays, e.g., displaypanels mounted on a wall. A missing feature in these applications usingDVIs is the capability to support user-aware, real-time or quasireal-time interactions in response to the user's position andorientation relative to the video display. Typically, conventionalwireless video display systems operate by transmitting compressed videosignals to the display, such that the latter requires decompressioncircuits, over license-free but narrow designated radio spectrum bands;however, these narrowband wireless systems can generally not support thedesired high data rates for the transmission of raw high-resolutionvideo signals. Furthermore, narrowband wireless systems are generallynot capable to support applications requiring a precise indoor locationtracking capability.

To mitigate the threat of a future spectrum shortage, additional radiospectrum in the form of the ultra-wideband (UWB) radio channel wasrecently made available for use in the USA in the range 3.1 GHz-10.6GHz. European and Asian authorities are also preparing rulings to enablecommercial marketing and use of devices based on UWB radio technology(UWB-RT).

High-definition television (HDTV) and high-resolution video displayrendering will use DVIs with the added disadvantage that hightransmission data rates are to be used between the video signal sourceand the display. For example, the digital DVI (DVI-D) standard linkallows a digital connection from a set-top box (STB) to a displayeliminating unnecessary digital to analog conversions and keeping thesignal digital. Almost all STBs use DVI-D and do not carry any analogsignal. An HDTV capable TV set has maximum resolution of 1920×1080pixels (60 Hz AC power supply) and should only use a single link (cableor wireless), as do almost all DVI equipped displays. However, plasmascreens and flat panel displays (FPDs) can have much higher resolutionsand refresh rates and could use a dual or multiple DVI-D links. Therequirement for very high data rates between the video signal sourcesand high-resolution displays is particularly demanding if the videosignal is to be transmitted wirelessly to the display. Thus, there is aneed for new solutions to increase both the effective data rate and thelink distance between the video source and the display. This problemprevails equally when connecting a personal computer (PC) to ahigh-resolution video display through DVIs or other digital interfaces.

One possibility to overcome this problem is to split a high-rate datastream into multiple lower-rate data streams to achieve a compound datarate equal to the desired high-data rate. In such cases, designers arefaced with the problem of choosing the best possible multi-streamtechnology for a given set of system parameters and criteria. Thereexist two different approaches: a) support of multiple data streams bymeans of co-located and non-interfering networks, also called piconets,and b) support of multiple data streams within a single communicationcell based on a suitable multiple access (MA) technique. All of theseapproaches are based on an efficient channel access method and the mostpopular prior art MA methods are briefly described hereafter, listingtheir known features and suitability when applied to networks withterminals based on UWB-RT.

MA techniques, e.g., Time Division Multiple Access (TDMA), FrequencyDivision Multiple Access (FDMA), Code Division Multiple Access (CDMA),and in particular Rate Division Multiple Access (RDMA) as described byMartin Weisenhorn and Walter Hirt, “Novel Rate-Division Multiple-AccessScheme for UWB-Radio-Based Sensor Networks”, IEEE Int. Zurich Seminar onCommunications, pages 76-81, February 2004, have been known for sometime. Some of these techniques could be applied to mitigate thehigh-rate video distribution problem. For example, when using RDMA, eachdata stream is transmitted by a binary antipodal signal, preferably witha low duty cycle, that can be distinguished at the destination receiverby an appropriate, unique choice of a user-specific pulse rate or acombination of at least two user-specific pulse rates. However, it maybe advantageous in certain applications to combine the features andbenefits of the basic RDMA method with one or more of the prior artmultiple access methods.

Known wireless (video) displays include:

-   -   PDA-like screens of low pixel resolution (320×240),        near/full-motion video (15/30 fps (frames per second)), in high        or true color format (12, 16 or 24 bit/pixel);    -   tablet—SVGA up to XGA resolution, 16/24 bit/pixel, with full or        reduced refresh rate; if full-motion video is supported, then        the video stream is commonly MPEG-encoded when sent over a        wireless channel, with data rates of 1-10 Mbps; and    -   video projectors and TV displays with internal MPEG decoding        engine and Wi-Fi® capability with nominal data rates of 11-54        Mbps (Wi-Fi is a registered trademark of the Wi-Fi Alliance).

While some of the above will support full-motion video in DVD or evenHDTV quality, none of these display devices can sustain thehigh-resolution full-rate refresh requirements of a standard PC over awireless channel. A distinction is made between a cinematographic and“full-motion” video display of 25, resp. 30 fps, and a more demandingnotion of “full-rate” computer monitor, with refresh rates in excess of60 fps. Here the requirements of full-rate display refresh areaddressed.

A question worth asking is: How to wirelessly connect a full-rate highresolution PC monitor? A brief step-wise calculation of the bandwidthrequirements yields the following results:

a) A typical PC monitor resolution ranges from XGA (1024×768) up toQUXGA-W (3840×2560); most common resolutions are SXGA/+(1280×1024) andUXGA (1600×1200). The large scale introduction of HDTV will drive thestandard PC resolution up to QXGA (2048×1536).

b) The minimum refresh rate for PC displays in TFT LCD technology is 60fps.

c) Unlike video codecs such as MPEG-2/4, there seem to be no generalcodecs for GUIs (Graphical User Interfaces). A universal GUI displaythat works with a common operating system uses the least commondenominator, i.e., un-encoded video transmission between the computervideo source and its display. Otherwise, the lack of a large enoughmarket would not allow the widespread use of full rate/motion wirelessdisplays.

Thus the net bandwidth required by an XGA display at 60 fps is:(1024×768 pix)×(24 bit/pix)×(60 fps)=1132 Mbps or 1.132 Gbps. Assuming a25% overhead for wireless encoding and protocol, the raw data rate forXGA is ˜1.4 Gbps; for QXGA: ˜5.7 Gbps; for QUXGA-W: ˜17.7 Gbps.

Considering SXGA/UXGA, the most likely resolution used for PC, notebookand HDTV flat screens, it is found that the raw data rates are also veryhigh for wireless transmission (SXGA: ˜2.4 Gbps; UXGA: ˜3.5 Gbps).

From the above it follows that there is still a need in the art for ascheme that allows displaying of a video signal on a display withoutcables and in particular depending upon a user interaction. It would befurther advantageous if the scheme enables a user-aware wireless videodisplay offering full rate refresh at high resolution.

SUMMARY OF THE INVENTION

Therefore, in accordance with the invention, there is provided a systemfor displaying a video signal depending upon a user interaction. Anexample of a system comprises: at least one transmitter to transmit avideo signal, a display for receiving the video signal, at least onetransmitter for transmitting location signals, and at least threetransceivers for receiving the location signals from the transmitter andfor transmitting modified location signals. The geometric locations ofthe at least three transceivers are known to a computing device. Thecomputing device is then able to derive a transmit location of thelocation signals based on the received modified location signals, whereafterwards the computing device is adapted to modify the content of thevideo signal in response to the derived transmit location depending uponthe user interaction. The user interaction changes or moves the transmitlocation of the location signals when, for example, a user moves thedisplay or himself.

In accordance with another aspect of the invention, there is provided amethod for displaying a video signal on a display depending upon a userinteraction. An example of a method comprises the steps of: receiving bythe display the video signal from a computing device; transmittinglocation signals via at least three transceivers to the computingdevice; receiving by the computing device modified location signals fromthe at least three transceivers, the geometric locations of the at leastthree transceivers being known to the computing device; deriving by thecomputing device a transmit location of the location signals based onthe received modified location signals; and modifying the content of thevideo signal in response to the derived transmit location depending uponthe user interaction.

DESCRIPTION OF FIGURES

Advantageous embodiments of the invention are described in detail below,by way of example only, with reference to the following schematicfigures, in which:

FIG. 1 a illustrates a first wireless multi-channel video distributionsystem with a set-top box (STB) and a flat panel display (FPD). In theSTB, the video source's data stream is divided into four parallel datasub-streams. Each receiver built into the display is at least capable ofreceiving one of the four sub-streams;

FIG. 1 b illustrates a further wireless multi-channel, high-resolutionvideo distribution system with a graphic and video sub-system (GVS) anda flat panel display (FPD). In the GVS, the video source's data streamis divided into four parallel data sub-streams. Generally, each receiverbuilt into the display and connected to the respective antenna (A1, A2,A3 or A4) is at least capable of receiving one of the four datasub-streams;

FIG. 2 a shows a basic system setup usable to support high-resolutionvideo signal distribution to an un-tethered FPD when the datasub-streams are sent from a GVS to wireless transceivers (TR1, TR2, TR3and TR4) by means of wires;

FIG. 2 b shows the basic system setup of FIG. 2 a supportinglocalization and tracking of one antenna (A1) of the FPD within thereference frame defined by the indicated (x,y,z) coordinate system withorigin O=(0,0,0). The position of the antenna A1 is determined by theposition vector of antenna A1 (PVA1) defined by the geometric positioncoordinates p₁=(x₁,y₁,z₁).

FIG. 2 c illustrates an extension of the setup shown in FIG. 2 b to allfour antennas of the FPD, where the position of antenna An is determinedby the position vector of antenna n (PVAn) as defined by the coordinatesp_(n)=(x_(n),y_(n),z_(n)), n=1,2,3,4 (for reasons of clarity, only PVA4is shown);

FIG. 3 a shows a further basic system setup usable to supporthigh-resolution video signal distribution to an un-tethered FPD when thedata sub-streams are sent to the wireless transceivers (TR1, TR2, TR3and TR4) by means of further wireless links, preferably based on UWB-RT;

FIG. 3 b shows the basic system setup of FIG. 3 a supportinglocalization and tracking of the spatial position of antenna A1 mountedon the FPD within a reference frame, defined by the indicated (x,y,z)coordinate system with origin O=(0,0,0) Assuming that the positions ofthe other three antennas (A2, A3 and A4) have been determined in asimilar way (e.g., see FIG. 2 c), this figure also illustrates how theposition and orientation of the FPD can be described by means of i) apanel position vector (PPV) pointing to the center of the display paneland ii) a perpendicular panel orientation vector (POV) that defines thegeometric orientation of the panel plane;

FIG. 4 a illustrates a further basic wireless multi-channel videodistribution system with a graphics and video subsystem (GVS) and a wallmounted, stationary flat panel display (FPD). In the GVS, the videosource's data stream is divided into four parallel data sub-streams thatare carried by cables to corresponding transceivers (TR1, TR2, TR3 andTR4). The latter transmit the data sub-streams either by means of i)four independent piconets or ii) by means of a multi-user communicationnetwork operating within a single communication cell. Generally, eachreceiver built into the display and connected to the respective antenna(A1, A2, A3 or A4) is at least capable of receiving one of the four datasub-streams;

FIG. 4 b illustrates another basic wireless multi-channel videodistribution system capable to support applications based on userinteraction. The system comprises a graphics and video subsystem (GVS)and a wall mounted flat panel display (FPD). In the GVS, the videosource's data stream is divided into four parallel data sub-streams thatare carried by cables to corresponding transceivers (TR1, TR2, TR3 andTR4). Generally, each receiver built into the display and connected tothe respective antenna (A1, A2, A3 or A4) is at least capable ofreceiving one of the four data sub-streams. The basic system is enhancedwith a transmitting beacon (B1) carried by the user that allowsmeasurement of the user's position and orientation relative to astationary display; and

FIG. 4 c illustrates the system of FIG. 4 b but with the difference thatthe wired links between the GVS and the transceivers (TR1, TR2, TR3,TR4) have been replaced by wireless links (for reasons of clarity, onlythe links between the GVS and TR1 and TR2 are shown).

The drawings are provided for illustrative purposes only.

DESCRIPTION OF THE INVENTION

This invention provides methods and systems for displaying a videosignal on a display depending upon a user interaction. Interestingapplications become feasible which are based on a user-aware wirelessvideo display offering full rate refresh at high resolution. The systemprovided is particularly well suited for wireless communication andpositioning technologies exploiting the ultra-wideband (UWB) radiochannel recently made available by the US Federal CommunicationsCommission (FCC) in the frequency range between 3.1 GHz and 10.6 GHz.

Thus, the present invention provides a system for displaying a videosignal depending upon a user interaction. An embodiment of a systemcomprises at least one transmitter to transmit a video signal, a displayfor receiving the video signal, at least one transmitter fortransmitting location signals, and at least three transceivers forreceiving the location signals from the transmitter and for transmittingmodified location signals. The geometric locations of the at least threetransceivers are known to a computing device. The computing device isthen able to derive a transmit location of the location signals based onthe received modified location signals, where afterwards the computingdevice is adapted to modify the content of the video signal in responseto the derived transmit location depending upon the user interaction.The user interaction changes or moves the transmit location of thelocation signals when, for example, a user moves the display or himself.In advantageous embodiments multiple transmitters for transmittinglocation signals are used in order to increase the accuracy of thelocation determination.

The invention also provides a method for displaying a video signal on adisplay depending upon a user interaction. The method comprises thesteps of: the display receiving the video signal from a computingdevice; transmitting location signals via at least three transceivers tothe computing device; the computing device receiving modified locationsignals from the at least three transceivers, the geometric locations ofthe at least three transceivers being known to the computing device; thecomputing device deriving a transmit location of the location signalsbased on the received modified location signals; and modifying thecontent of the video signal in response to the derived transmit locationdepending upon the user interaction.

The method can further comprise the steps of: each of the at least threetransceivers receiving the location signals, determining arrival timesfrom the received location signals, and each of the at least threetransceivers transmitting the determined arrival times within modifiedlocation signals to the computing device. This allows to provide thecomputing device with the relevant information for the subsequentdetermination of the transmit location.

When the display is a mobile or portable, tablet-like user-awarewireless display or flat panel display (FPD) that is freely movable by auser, then the user can decide by user interaction how something isshown or from which angle, e.g., of a picture, movie, or construction,it is displayed.

The position (location) of the display can be fixed or the display canmounted on a wall, and then the location signals can be transmitted by abeacon transmitter that can be attached to a user. Thus, the calculatedtransmit location of the location signals is directly related to themovement of the user. The fixed or mounted display can then be astationary large size user-aware display usable for immersive virtualreality or other applications.

The connection between the computing device and the display can beestablished by cables (partly), various wireless links, or a combinationthereof. Best suited for the wireless links are UWB radio or other verywideband RF (radio frequency) technologies. The transmission of thevideo signal from the computing device to the display and thetransmission of the location and modified signals to the computingdevice via the at least three transceivers can be performed by wirelesslinks. Then, no cables are required, which allows more freedom ofmovement for the user.

As indicated, user awareness is enabled through two basic approaches: i)by determining the geometric position and orientation of the displaywith the transceivers, e.g., of a handheld wireless display panel,carried by a user within a reference frame; ii) by determining a user'sgeometric position and orientation in front of a stationary wirelessdisplay, while the user has a beacon transmitter attached.

When the display is stationary, e.g. mounted on a wall, then thegeometric locations of the display are known to the computing device.The computing device is then able to derive the transmit location of thebeacon transmitter relative to the display's geometric location based onthe received modified location signals and to modify the content of thevideo signal in response to the derived transmit location of the beacontransmitter depending upon the user interaction, i.e. movement of thebeacon transmitter.

The video signal or channel can be split into at least threesub-channels, which can be realized, for example, with UWB radiotechnology, based on coordinated piconets. The at least threesub-channels can be de-multiplexed by a Graphical Processing Unit (GPU),or by a standard chipset at the read port of the video RAM. That means ahigh-rate video data stream can be split into multiple sub-streams, suchthat each sub-stream can be transmitted wirelessly to a full-ratehigh-resolution display, where the sub-streams are combinable into acomposite video data stream (video signal). A suitable arrangement ofthe transceivers and the antennas associated with the display enableuser-aware applications. The wireless display can featurefull-rate/full-motion refresh at high resolutions.

At the receiving end, the display will process the video sub-channels byallocating and mapping each of them into one of the adjacent verticalstripes. Whereas the mapping to the actual screen circuitry can be donedifferently, a relevant feature is the geometrical location of thetransmitter, as this defines the transmit location of the locationsignals. This location should be eligible or appropriate for wirelesstriangulation.

Based on today's technological developments and the assumption thatfurther advanced state-of-the-art UWB wireless links will becomeavailable in the future, it is estimated that three or four parallelchannels for SXGA and SXGA+ displays will be suitable to achieve acompound data rate of well above 1 Gbps over an acceptable distance.

The desire to split a high-rate video data stream into multiplesub-streams, such that each sub-stream can be transmitted wirelessly tothe same destination, e.g., to a high-resolution display, can beexploited further and thus has further advantages. An implementation ofa suitable arrangement of receiving antennas on the display can be usedto enable user-aware applications or interactions, such that the usercan interact with graphic or video shown on said display. Below arelisted some advantages relating to displays, particularlyhigh-resolution video displays, which are capable to supportapplications based on user awareness and/or interaction:

-   -   a triangulation-based localization—which enables user-awareness,        virtual reality (VR) etc.—adds only a minor cost to the display,        this comes as a by-product of having a full-rate high-resolution        wireless display;    -   in the future, the implementation is possible with        commercial-off-the-shelf (COTS) components;    -   wide range of applications, from scientific and engineering,        design, business, e-learning, to consumer and entertainment;    -   potential products can be marketed both as commodity h/w and as        dedicated, professional systems;    -   the affordability of implementations lowers the entry barriers        of established virtual reality (VR) applications and may enable        new, un-anticipated applications;    -   transmission of multiple sub-streams of data enables high        aggregate data rates suitable for high-resolution video signal        display;    -   suitable placement of multiple antennas on the display, that can        be a flat panel display (FPD), can be exploited to enable        applications desiring user-aware interaction;    -   the combination of multiplexed, full-rate high-resolution        wireless video streaming to a movable display and simultaneous        wireless 3D position location and tracking of the display        enables novel user-aware applications, one such application is        the portable tablet usable for virtual reality (VR) and 3D        interaction, the innate hand-eye coordination property of the        human sensorial system (retina>cortex>motor nerve>muscle) can be        exploited;    -   expensive electro-mechanics are replaced by wireless        triangulation (i.e., position location and tracking);    -   the combination of multiplexed, full-rate high-resolution        wireless video streaming to a fixed display and simultaneous        wireless 3D position location and tracking of a user in front of        the fixed display enables novel user-aware applications, e.g.,        for engineers or designers,    -   such an application is a large, wall-mounted user-aware display,        where the same type of triangulation scheme enables an        “immersive VR”-class of applications, without the need of wiring        the users with intrusive sensors;    -   the precision of the 3D positioning can be optimized by proper        geometric placement of the antennas on the display, e.g., by        maximizing the spatial separation between antennas and mounting        them in the same plane;    -   the precision of the 3D positioning can be further optimized by        proper geometric placement of the at least three transceivers        within the room where the display is used, e.g., in order to        obtain minimal GDP (geometric dilution of precision), thus        avoiding that all transceivers are placed within a plane;    -   3D positioning is preferably based on a TDOA (time difference of        arrival) scheme.

In addition, the proposed space-multiplexed wireless video transmissionscheme provides and supports:

-   -   multi-purpose video sources: from PCs to game consoles, DVD        players, HDTV tuners, video-cams, video servers;    -   multi-purpose displays: the same wireless panel(s) could be used        as tablet, monitor or HDTV display;    -   mobility within a range of up to 4 m, or farther at reduced        frame rate;    -   a low cost wireless system that consists of four COTS chipsets,        e.g., adapted from the USB2.0 or other connecting interfaces;        and    -   reduced costs of new applications and by-products, such as        affordable virtual reality (VR).

Of particular interest are the user-aware displays and “immersive VR”systems, when built on commodity hardware. Such systems can be used forscientific and engineering visualization, remote learning, telemedicine,conferencing systems, e-retail, arcade and multi-user/Net-based games.

FIG. 1 a illustrates a wireless multi-channel video distribution system1 comprising a set-top box (STB) 10 and a flat panel display (FPD) 30which are connected via wireless links 5. The set-top box 10 comprises avideo source interface 11, an STB Controller & De-Multiplexer unit 12,and STB transmit/receive units 13 that are connected to STB antennas 14and the STB Controller & De-Multiplexer unit 12, respectively. The STBController & De-Multiplexer unit 12 receives the video signal from thevideo source interface 11 that is connected to a computing device 50that provides the video signal, also referred to as video data stream,via a video link 7. The computing device 50 receives modified locationsignals via an information link 8 from the STB Controller &De-Multiplexer unit 12.

The flat panel display (FPD) 30 comprises a video sink unit 31, an FPDController & Multiplexer unit 32, and display transmit/receive units 33that are connected to FDP antennas 34 and the FPD Controller &Multiplexer unit 32, respectively. The FPD Controller & Multiplexer unit32 provides the received video signal to the sink unit 31 for furtherprocessing and displaying.

In the set-top box 10, the video data stream coming from the computingdevice 50 is here divided into four parallel data sub-streams to feed i)four independent piconets or ii) a multi-user communication networkoperating within a single communication cell, where each displaytransmit/receive unit 33 built into the display 30 is at least capableof receiving one of the four sub-streams. When receiving one of the foursub-streams each display transmit/receive unit 33 together with itscorresponding FDP antenna 34 is also referred to as display receiver. Inthis setup, the set-top box 10 transmits preferably un-compressed videosignals in each sub-channel to remove the burden of signal decompressionin the display 30, thereby reducing power consumption and cost.

Similarly, FIG. 1 b illustrates a wireless multi-channel,high-resolution video distribution system 2 comprising a graphic andvideo sub-system (GVS) 20 and the flat panel display 30. The samereference numbers are used to denote the same parts or the like. Thegraphic and video sub-system 20, short GVS 20, comprises ahigh-resolution video source interface 21, a GVS Controller & VideoDe-Multiplexer unit 22, and GVS transmit/receive units 23 that areconnected to GVS antennas 24 and the GVS Controller & VideoDe-Multiplexer unit 22, respectively. The GVS Controller & VideoDe-Multiplexer unit 22 receives a high-resolution video signal from thecomputing device 50 via the video link 7 and the high-resolution videosource interface 21. On the other hand, the computing device 50 receivesmodified location signals via the information link 8 from the GVSController & Video De-Multiplexer unit 22.

In the graphic and video sub-system 20, the high-resolution video datastream coming from the computing device 50 is divided by the GVSController & Video De-Multiplexer unit 22 into four parallel datasub-streams to feed i) four independent piconets or ii) a multi-usercommunication network operating within a single communication cell.Generally, each display transmit/receive unit 33 built into the display30 and connected to the respective FDP antenna 34 (A1, A2, A3 or A4) isat least capable of receiving one of the four data sub-streamstransmitted via the GVS antennas 24. A1, A2, A3 and A4 indicate thetransmit location of the respective FDP antenna 34 of the display 30.The positions of the FPD antennas 34 at the display 30 and particularlythe distances between the antennas are a priori known by design. Forboth embodiments indicated in FIGS. 1 a and 1 b, the four FPD antennas34 and respective transmit/receive units 33 offer three different modesof operation that can be used separately or in combination:

High Data Rate Video Streaming

Aggregation of the four parallel sub-system data streams to obtain afour-fold compound data rate enabling high rate video streaming to thedisplay 30. This mode solves the problem that a single wireless link 5may not be able to sustain the desired high compound data rate over thepreferred distance.

Improved Link Range and/or Link Robustness

The set-top box 10 and the flat panel display 30 form amultiple-input/multiple-output (MIMO) transmission channel, where theset-top box 10 emits the same video source data over all sub-channels,thereby introducing a spatial diversity effect that can be exploited.Correspondingly, all four FPD antennas 34 and display transmit/receiveunits 33 receive up to four versions of the same video source datastream. This allows the display system to exploit the advantages of MIMOcommunication by i) achieving a higher data rate over a given linkdistance or ii) achieving an increased link distance for a given lowerdata rate or iii) achieving a higher link robustness or iv) using anycombination of i) to iii).

Position Location and Tracking

With each of the four independent FPD antennas 34 placed in one of thecorners of the display 30 (maximum spatial separation) the display 30can be enabled with user-awareness by sensing the geometric position ofthe four FPD antennas 34. From knowledge of the positions of at leastthree of these antennas 34 the orientation of the panel's viewing planeand its absolute position within a reference frame can be determined;such information is useful for graphic rendering applications requiringuser awareness for interactive display operation and feedback. Furtherdetails of this operational mode will be discussed below.

FIGS. 2 a, 2 b and 2 c illustrate a basic system setup in an office orresidential environment with wireless transceivers 41, 42, 43, 44, alsoreferred to as transceivers 41-44. FIG. 2 a shows the setup usable tosupport high-resolution video signal distribution, over foursub-channels (C1-C4) to an un-tethered display 30 when the video data issent to the wireless transceivers 41, 42, 43, 44, also labeled with TR1,TR2, TR3 and TR4, by means of cables or cable links 4. Depending on theapplication, the wireless links 5 between the transceivers 41, 42, 43,44 and the display 30 may be unidirectional or bidirectional, asindicated in the figure. In this first arrangement where the primaryapplication is to feed a video signal to the display 30, the actualpositions of the transceivers 41-44 is not critical as long as thedesired quality of service (QoS) can be maintained over the wirelesslinks 5.

FIG. 2 b illustrates how the video streaming system setup can be used tosupport localization and tracking of one FDP antenna 34, here alsoreferred to as antenna A1, of the display 30 within a reference framedefined by the indicated (x,y,z) coordinate system with originO=(0,0,0). The respective location or position of the antennas areindicated by and labeled with An, with n=1,2,3,4. As indicated in thefigure, the position of antenna A1 (e.g., the antenna's feed point maybe used to define its position) is determined by the position vector ofantenna A1 (PVA1) defined by the geometric coordinates p₁=(x₁,y₁,z₁),which can be determined by the following procedure. For reasons ofsimplicity, detailed descriptions are only given for antenna A1; similarprocedures can be applied to the other FDP antennas, labeled with A2, A3and A4 as indicated in FIG. 2 c.

1) Antenna A1 emits a so-called positioning signal (PS), e.g., in theform of an acknowledgment (ACK) signal after reception of a video datapacket, that is receivable by each of the four transceivers 41-44 (TR1,TR2, TR3 and TR4) and that comprises unambiguous information to identifyit as a signal emitted from antenna A1. The time instant of thisemission is not known and thus not available anywhere in the system.

2) The transceivers 41-44 (TR1, TR2, TR3 and TR4) register reception ofthe PS from antenna A1 and relay the source identification informationand a time count to the GVS 20 in the manner as described in the UnitedStates Patent Application 20030174086, presently assigned to theassignee of the instant application and the disclosure of which isincorporated herein by reference. Alternatively, the transceivers 41-44have the capability to measure the absolute arrival time of the PS sentby antenna A1, in which case the transceivers 41-44 forward the sourceidentification information and the absolute arrival time to the GVS 20within the modified location signals.

3) Given the known propagation times between each transceiver 41, 42,43, 44, and the GVS 20, a processing unit (not shown) in the computingdevice 50 first computes the time difference of arrival (TDOA) betweenthree pairs of received transceiver signals. Alternatively, the TDOAvalues can be computed from measured absolute arrival times, which aresent by transceivers 41-44 via the GVS 20 to the processing unit(computing device) 50. This procedure yields sufficient information tofind a solution based on known methods for the position vector ofantenna A1 (PVA1): p₁=(x₁,y₁,z₁).

4) The above procedure can be repeated in parallel or sequentially foreach FPD antenna 34 of the display 30 to yield position vectors for eachFPD antenna 34: p_(n)=(x_(n),y_(n),y_(z)) n=1,2,3,4. Any three of thesevectors can be used to define the orientation of the FPD 30 plane whilethe fourth redundant position vector may be used to improve the plane'sparameter estimates (see also FIG. 3 b).

5) The information of the FPD plane orientation within the referenceframe can be used by the application that renders the video signal fordisplay.

A further embodiment of the proposed system is indicated in FIG. 3 a,showing the basic system setup usable to support high-resolution videosignal distribution to the un-tethered display 30 when the datasub-streams are sent to the wireless transceivers 41, 42, 43, 44 (TR1,TR2, TR3 and TR4) by means of further wireless links 6, therebyreplacing the cable links 4 illustrated in FIG. 2 a.

Consequently, FIG. 3 b shows an alternative to the embodimentillustrated in FIG. 2 b, where FIG. 3 b indicates how the orientation ofthe FPD plane can be defined after at least three of the four antennapositions (A1, A2, A3, A4) within the FPD plane have been determined. Inparticular, the position and orientation of the FPD plane is defined byi) the panel position vector (PPV) pointing to the geometric center ofthe FPD plane and ii) the plane orientation vector (POV) positionedperpendicular to the FPD plane; without loss of generality, the POV maybe a normalized unit vector. The coordinates of the PPV and the POV,derivable from the TDOA measurements described above, provide sufficientinformation for an application program rendering video signals inresponse to the FPD's geometric position and orientation within thegeometric reference frame defined by the (x,y,z) coordinate system withorigin O=(0,0,0).

In operation, the content of a video signal on the display 30 ismodified depending upon a user interaction, i.e., when a user moves thedisplay 30. Generally, the display 30 receives the video signal from thecomputing device 50.

Location signals are transmitted from the FDP antennas 34 via the atleast three transceivers 41-44 and the GVS 20 to the computing device50. In the transceivers 41-44, whose geometric locations are also knownto the computing device 50, the received location signals aretransformed to modified location signals which then include arrival timeinformation. The computing device 50 that then receives the modifiedlocation signals from the transceivers 41-44 is able to derive eachtransmit location (A1, A2, A3, A4) of the location signals based on thereceived modified location signals. The content of the video signal isthen modified by the computing device 50 in response to the derivedtransmit location (A1, A2, A3, A4) depending upon the user interaction,that is the movement of the display 30 in all directions.

A further application of the display 30 is illustrated in FIG. 4 a,where the display 30 remains in a stationary position, for example bymounting it on a wall. Depicted is a wireless multi-channel videodistribution system with the GVS 20 and the display 30 that here is awall mounted flat panel display 30. In the GVS 20, the video source'sdata stream is divided into four parallel data sub-streams (C1-C4) thatare carried by cable links 4 to the respective transceivers 41-44. Thelatter transmit the data sub-streams via the wireless links 5 either bymeans of i) four independent piconets or ii) by making use of amulti-user communication network operating within a single communicationcell. Generally, each display transmit/receive unit 33 built into thedisplay 30 and connected to the respective antenna 34 is at leastcapable of receiving one of the four data sub-streams via the wirelesslinks 5.

FIG. 4 b illustrates further the basic wireless multi-channel videodistribution system introduced in FIG. 4 a to support applications basedon user interaction. The system comprises again the GVS 20 and the wallmounted flat panel display, i.e., the display 30. In the GVS 20, thevideo source's data stream is divided into four parallel datasub-streams (C1-C4) that are carried by the cables links 4 to thecorresponding transceivers 41-44. The latter transmit the datasub-streams either by means of i) four independent piconets or ii) bymaking use of a multi-user communication network operating within asingle communication cell. Generally, each display transmit/receive unit33 built into the display 30 and connected to the respective antenna 34(with position A1, A2, A3 or A4) is at least capable of receiving one ofthe four data sub-streams. The system is enhanced with a transmittingbeacon 60, also referred to as beacon transmitter 60, carried by a user62. The place of the beacon transmitter 60 defines here the transmitlocation. This arrangement allows measurement of the user's positionp_(u)=(x_(u),y_(u),z_(u)) and his orientation with respect to thestationary display 30. In this setup, the display's position is knownand defined as its geometric center with the coordinatesp_(d)=(x_(d),y_(d),z_(d)), thus the user direction vector (UDV),indicating a user's position relative to the FPD's known position andorientation, is obtained by the vector differencep_(UVD)=p_(u)−p_(d)=(x_(u)−x_(d),y_(u)−y_(d),z_(u)−z_(d)). Thecoordinates of the UVD, p_(u)=(x_(u),y_(u),z_(u)), are derivable fromthe TDOA measurements as described above; together with the display'sknown position and geometric orientation, p_(d)=(x_(d),y_(d),z_(d)),sufficient information can be provided to an application programrendering video signals in response to the user's geometric positionthat is the transmit location of the beacon transmitter 60, indicatedwith B1. Thus, the user's location within the geometric reference framedefined by the (x,y,z) coordinate system with origin O=(0,0,0) can bedetermined. The application program that modifies the video signalexecutes on the computing device 50 which for simplicity is not shown inthe figure. The computing device 50 can be any computer or sever that isable to influence the video signal based on the calculations.

In operation, the display 30 receives the video signal from thecomputing device 50 via the GVS 20 and the transceivers 41-44. Asindicated, the geometric location and orientation of the display 30 isknown. Location signals are transmitted by the beacon transmitter 60 viathe transceivers 41-44 and the GVS 20 to the computing device 50. In thetransceivers 41-44, whose geometric locations are also known to thecomputing device 50, the received location signals are transformed tomodified location signals which then include arrival time information.The computing device 50 that then receives the modified location signalsfrom the transceivers 41-44 is able to derive a transmit location (B1)of the location signals relative to the display's geometric locationbased on the received modified location signals. The content of thevideo signal is then modified by the computing device 50 in response tothe derived transmit location (B1) depending upon the user interactionby the user 62.

Alternatively, in the setup of FIG. 4 b, it is not necessary to know thedisplay's position a priori. Similarly to the embodiment illustrated inFIG. 3 b, it is possible to determine the display position coordinatesp_(d)=(x_(d),y_(d),z_(d)) by measuring TDOA parameters and to computethe display position coordinates from location signals transmitted viathe FPD's antennas (A1-A4).

A further alternative to the system shown in FIG. 4 b is indicated inFIG. 4 c, where the cable links 4 between the GVS 20 and thetransceivers 41-44 have been replaced by further wireless links 6, i.e.,the four data sub-streams (C1-C4) are sent wirelessly to the referencetransceivers 41-44. Variations described for the present invention canbe realized in any combination desirable for each particularapplication. Thus particular limitations, and/or embodiment enhancementsdescribed herein, which may have particular advantages to a particularapplication need not be used for all applications. Also, not alllimitations need be implemented in methods, systems and/or apparatusincluding one or more concepts of the present invention. Methods may beimplemented as signal methods employing signals to implement one or moresteps. Signals include those emanating from the Internet, etc.

The present invention can be realized in hardware, software, or acombination of hardware and software. A visualization tool according tothe present invention can be realized in a centralized fashion in onecomputer system, or in a distributed fashion where different elementsare spread across several interconnected computer systems. Any kind ofcomputer system—or other apparatus adapted for carrying out the methodsand/or functions described herein—is suitable. A typical combination ofhardware and software could be a general purpose computer system with acomputer program that, when being loaded and executed, controls thecomputer system such that it carries out the methods described herein.The present invention can also be embedded in a computer programproduct, which comprises all the features enabling the implementation ofthe methods described herein, and which—when loaded in a computersystem—is able to carry out these methods.

Computer program means or computer program in the present contextinclude any expression, in any language, code or notation, of a set ofinstructions intended to cause a system having an information processingcapability to perform a particular function either directly or afterconversion to another language, code or notation, and/or reproduction ina different material form.

Thus the invention includes an article of manufacture which comprises acomputer usable medium having computer readable program code meansembodied therein for causing a function described above. The computerreadable program code means in the article of manufacture comprisescomputer readable program code means for causing a computer to effectthe steps of a method of this invention. Similarly, the presentinvention may be implemented as a computer program product comprising acomputer usable medium having computer readable program code meansembodied therein for causing a function described above. The computerreadable program code means in the computer program product comprisingcomputer readable program code means for causing a computer to affectone or more functions of this invention. Furthermore, the presentinvention may be implemented as a program storage device readable bymachine, tangibly embodying a program of instructions executable by themachine to perform method steps for causing one or more functions ofthis invention.

It is noted that the foregoing has outlined some of the more pertinentobjects and embodiments of the present invention. This invention may beused for many applications. Thus, although the description is made forparticular arrangements and methods, the intent and concept of theinvention is suitable and applicable to other arrangements andapplications. It will be clear to those skilled in the art thatmodifications to the disclosed embodiments can be effected withoutdeparting from the spirit and scope of the invention. The describedembodiments ought to be construed to be merely illustrative of some ofthe more prominent features and applications of the invention. Otherbeneficial results can be realized by applying the disclosed inventionin a different manner or modifying the invention in ways known to thosefamiliar with the art.

1. A display method comprising displaying a video signal on a displaydepending upon user interaction, the step of displaying comprising thesteps of: the display receiving the video signal from a computingdevice; transmitting location signals from at least one transmitter viaat least three transceivers to the computing device; the computingdevice receiving modified location signals from said at least threetransceivers having geometric locations known to the computing device;the computing device deriving a transmit location of the locationsignals based on the received modified location signals; and modifyingcontent of the video signal in response to the derived transmit locationdepending upon the user interaction.
 2. The method according to claim 1,further comprising the steps of: each of said at least threetransceivers receiving the location signals, determining arrival timesfrom the received location signals, and each of said at least threetransceivers transmitting the determined arrival times within modifiedlocation signals to the computing device.
 3. The method according toclaim 1, further comprising employing wireless links in connecting thecomputing device to the display.
 4. The method according to claim 1,wherein the display is a fixed display, and further comprising a step oftransmitting the location signals by a beacon transmitter.
 5. The methodaccording to claim 4, wherein the beacon transmitter is attached to auser.
 6. The method according to claim 1, wherein the video signal iscomprised of a video data stream, and further comprising splitting thevideo signal into multiple sub-streams, such that each sub-stream istransmittable wirelessly to the display, and the sub-streams arecombinable into a composite video data stream.
 7. The method accordingto claim 1, wherein the display is one of: a portable and a mobileuser-aware display freely movable by a user, and further comprising theuser deciding by user interaction how an object is shown.
 8. The methodaccording to claim 1, wherein a connection between the computing deviceand the display is established by at least one of: cables; variouswireless links; UWB radio, and very wideband radio frequencytechnologies.
 9. The method according to claim 1, wherein the videosignal is split into at least three sub-channels employing UWB radiotechnology based on coordinated piconets, and further comprisingde-multiplexing said at least three sub-channels employing one of: aGraphical Processing Unit and a standard chipset at the read port of thevideo RAM.
 10. The method according to claim 9, further comprising thedisplay processing the video sub-channels by allocating and mapping eachof them into one of the adjacent vertical stripes.
 11. An article ofmanufacture comprising a computer usable medium having computer readableprogram code means embodied therein for causing display of a videosignal, the computer readable program code means in said article ofmanufacture comprising computer readable program code means for causinga computer to effect the step of displaying the video signal on adisplay depending upon user interaction, the step of displayingcomprising the steps of: the display receiving the video signal from acomputing device; transmitting location signals from at least onetransmitter via at least three transceivers to the computing device; thecomputing device receiving modified location signals from said at leastthree transceivers having geometric locations known to the computingdevice; the computing device deriving a transmit location of thelocation signals based on the received modified location signals; andmodifying content of the video signal in response to the derivedtransmit location depending upon the user interaction.
 12. An article ofmanufacture as recited in claim 11, the computer readable program codemeans in said article of manufacture further comprising computerreadable program code means for causing a computer to effect the stepsof: each of said at least three transceivers receiving the locationsignals, determining arrival times from the received location signals,and each of said at least three transceivers transmitting the determinedarrival times within modified location signals to the computing device.13. An article of manufacture as recited in claim 11, the computerreadable program code means in said article of manufacture furthercomprising computer readable program code means for causing a computerto effect the step of employing wireless links in connecting thecomputing device to the display.
 14. An article of manufacture asrecited in claim 11, wherein the display is a fixed display, and thecomputer readable program code means in said article of manufacturefurther comprising computer readable program code means for causing acomputer to effect the step of transmitting the location signals by abeacon transmitter.
 15. A display system comprising: a display forreceiving the video signal, said system for displaying a video signaldepending upon a user interaction at least one transmitter fortransmitting location signals; at least three transceivers for receivingthe location signals from said at least one transmitter and fortransmitting modified location signals, the geometric locations of saidat least three transceivers being known; and a computing device forderiving a transmit location of the location signals based on thereceived modified location signals, wherein the computing device isadapted to modify the content of the video signal in response to thederived transmit location depending upon the user interaction.
 16. Thedisplay system according to claim 15, wherein said at least threetransceivers are adapted to determine arrival times from the receivedlocation signals, the determined arrival times are transmittable by saidat least three transceivers within the modified location signals to thecomputing device.
 17. The display system according to claim 15, whereinthe display is one of: a mobile display and a portable display.
 18. Thedisplay system according to claim 15, further comprising wireless linksfor the transmission of the video signal from the computing device tothe display and the transmission of the location and modified signals tothe computing device via said at least three transceivers.
 19. Acomputer program product comprising a computer usable medium havingcomputer readable program code means embodied therein for causingdisplay of a video signal, the computer readable program code means insaid computer program product comprising computer readable program codemeans for causing a computer to effect the functions of: receiving for adisplay the video signal, said display for displaying the video signaldepending upon a user interaction; at least one transmitter fortransmitting location signals; at least three transceivers for receivingthe location signals from said at least one transmitter and fortransmitting modified location signals, the geometric locations of saidat least three transceivers being known; and a computing device forderiving a transmit location of the location signals based on thereceived modified location signals, wherein the computing device isadapted to modify the content of the video signal in response to thederived transmit location depending upon the user interaction.
 20. Acomputer program product as recited in claim 19, the computer readableprogram code means in said computer program product further comprisingcomputer readable program code means for causing a computer to effectthe function of wireless links for the transmission of the video signalfrom the computing device to the display and the transmission of thelocation and modified signals to the computing device via said at leastthree transceivers.